Learning with Multiple Similarity Functions

Kernel functions have become an extremely popular tool in machine learning, with many applications and an attractive theory [1, 12, 10]. There has also been substantial work on learning kernel functions from data [7, 11, 2]. A sufficient condition for a kernel to allow for good generalization on a given learning problem is that it induce a large margin of separation between positive and negative classes in its implicit space. In recent work [4, 5, 3] we have developed a theory that more broadly holds for general similarity functions that are not necessarily legal kernel functions. In particular, we have introduced a notion of a good similarity function for a given learning problem that (a) is fairly natural and intuitive (it does not require an implicit space and allows for functions that are not positive semi-definite), (b) is a sufficient condition for learning well, and (c) strictly generalizes the notion of a large-margin kernel function in that any such kernel is also a good similarity function, though not necessarily vice-versa. We also have partial progress on extending the theory of learning with multiple kernel functions to this more general notion. In this note, we describe the main definitions and results of [4], give our results on learning with multiple similarity functions, and present several open questions.